Variational Belief Networks for Approximate Inference

Exact inference in large, densely connected probabilistic networks is computa-tionally intractable, and approximate schemes are therefore of great importance. One approach is to use mean eld theory, in which the exact log-likelihood is bounded from below using a simpler approximating distribution. In the standard mean eld theory, the approximating distribution is factorial. We propose instead to use a (tractable) belief network as an approximating distribution. The resulting compact framework is analogous to standard mean eld theory and no additional bounds are required, in contrast to other recently proposed extensions. We derive mean eld equations which provide an eecient iterative algorithm to optimize the parameters of the approximating belief network. Simulation results indicate a considerable improvement on previously reported methods.